Information-Theoretic Limits of Control 
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Fundamental limits on the controllability of physical systems are discussed in the light of infor- 
mation theory. It is shown that the second law of thermodynamics, when generalized to include 
information, sets absolute limits to the minimum amount of dissipation required by open-loop con- 
trol. In addition, an information-theoretic analysis of closed-loop control shows feedback control to 
be essentially a zero sum game: each bit of information gathered directly from a dynamical systems 
by a control device can serve to decrease the entropy of that system by at most one bit additional 
to the reduction of entropy attainable without such information (open- loop control). Consequences 
for the control of discrete binary systems and chaotic systems are discussed. 



PACS numbers: 05.45.+b, 05.20.-y, 89.70.+C 

Information and uncertainty represent complementary 
aspects of control. Open-loop control methods attempt 
to reduce our uncertainty about system variables such as 
position or velocity, thereby increasing our information 
about the actual values of those variables. Closed-loop 
methods obtain information about system variables, and 
use that information to decrease our uncertainty about 
the values of those variables. Although the literature in 
control theory implicitly recognizes the importance of in- 
formation in the control process, information is rarely 
regarded as the central quantity of interest In this 
Letter we address explicitely the role of information and 
uncertainty in control processes by presenting a novel for- 
malism for analyzing these quantities using techniques 
of statistical mechanics and information theory. Specif- 
ically, based on a recent proposal by Lloyd and Slotine 
[0], we formulate a general model of control and inves- 
tigate it using entropy- like quantities. This allows us to 
make mathematically precise each part of the intuitive 
statement that in a control process, information must 
constantly be acquired, processed and used to constrain 
or maintain the trajectory of a system. Along this line, 
we prove several limiting results relating the ability of 
a control device to reduce the entropy of an arbitrary 
system in the cases where (i) such a controller acts inde- 
pendently of the state of the system (open-loop control) , 
and (ii) the control action is influenced by some infor- 
mation gathered from the system (closed- loop control). 
The results are applied both to the stochastic example 
of coupled Markovian processes and to the deterministic 
example of chaotic maps. These results not only com- 
bine concepts of dynamical entropy and information in a 
unified picture, but also prove to be fundamental in that 
they represent the ultimate physical limitations faced by 
any control systems. 

The basic framework of our present study is the fol- 
lowing. We assign to the physical plant X we want to 
control a random variable X representing its state vec- 



tor (of arbitrary dimension) and whose value x is drawn 
according to a probability distribution p(x). Physically, 
this probabilistic or ensemble picture may account for in- 
teractions with an unknown environment, noisy inputs, 
or unmodelled dynamics; it can also be related to a de- 
terministic sensitivity to some parameters which make 
the system effectively stochastic. The recourse to a sta- 
tistical approach then allows the treatment of both the 
unexpectedness of the control conditions and the dynam- 
ical stochastic features as two faces of a single notion: 
uncertainty. 

As it is well known, a suitable measure quantifying un- 
certainty is entropy For a classical system with a 
discrete set of states with probability mass function p(x), 
it is expressed as 



H(X) = -J2 p(x)logp(x) 



(1) 



(all logarithms are assumed to the base 2 and the entropy 
is measured in bits). Other similar expressions also ex- 
ist for continuous state systems (fine-grained entropy), 
quantum systems (von Neumann entropy), and coarse- 
grained systems obtained by discretization of continuous 
densities in the phase space by means of a finite par- 
tition. In all cases, entropy offers a precise measure of 
disorderliness or missing information by characterizing 
the minimum amount of resources (bits) required to en- 
code unambiguously the ensemble describing the system 
J5| . As for the time evolution of these entropies, we know 
that the fine-grained (or von Neumann) entropy remains 
constant under volume-preserving (unitary) evolution, a 
property closely related to a corollary of Landauer's prin- 
ciple Q which asserts that only one-to-one mappings of 
states, i.e., reversible transformation preserving informa- 
tion are exempt of dissipation. Coarse-grained entropies, 
on the other hand, usually increase in time even in the ab- 
sence of noise. This is due to the finite nature of the par- 
tition used in the coarse-graining which, in effect, blurs 
the divergence of sufficiently close trajectories, thereby 
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inducing a "randomization" of the motion. For many 
systems, the typical average rate of this increase is given 
by a dynamical invariant known as the Kolmogorov-Sinai 
entropy @-|. 

In this context, we now address the problem of how 
a control device can be used to reduce the entropy of a 
system or to immunize it from sources of entropy, in par- 
ticular those associated with noise, motion instabilities, 
incomplete specification of states, and initial conditions. 
Although the problem of controlling a system requires 
more than limiting its entropy, the ability to limit en- 
tropy is a prerequisite to control. Indeed, the fact that a 
control process is able to localize a system in definite sta- 
ble states or trajectories simply means that the system 
can be constrained to evolve into states of low entropy 
starting from states of high entropy. 

To illustrate, in its most simple way, how the entropy 
of a system can be affected by external systems, let us 
consider a basic model consisting of our system X cou- 
pled to an environment £. For simplicity, and without 
loss of generality, we assume that the states of X form a 
discrete set. The initial state is again distributed accord- 
ing to p(x) 7 and the effect of the environment is taken 
into account by introducing a perturbed conditional dis- 
tribution p(x'\e), where x' is a value of the state later 
in time and e, a particular realization of the stochastic 
perturbation appearing with probability p(e). For each 
value e, we assume that X undergoes a unique evolution, 
referred here to as a subdynamics, taken to be entropy 
conserving in analog to the Hamiltonian time evolution 
for a continuous physical system: 



H(X'\e) = - 5>0z'|e) logp(x'\e) = H(X). 



(2) 



After the time transition X —> X' , the distribution p(x') 
is obtained by tracing out the variables of the environ- 
ment, and is used to calculate the change of the entropy 
H(X') — H(X) + AH. From the concavity property of 
entropy, it can be easily shown that AH > 0, with equal- 
ity if and only if (iff) the state £ is perfectly specified, 
i.e., if a value e appears with probability one. In prac- 
tice, however, the environment degrees of freedom are 
uncontrollable and the uncertainty associated with the 
environment coupling can be suppressed by "updating" 
somehow our knowledge of X after the evolution. One 
direct way to reveal that state is to imagine a measure- 
ment apparatus A coupled to X in such a way that the 
dynamics of the composed system X+£ is left unaffected. 
For this measurement scheme, the outcome of some dis- 
crete random variable A of the apparatus is described by 
a conditional probability matrix p(a\x') and the marginal 
p{a) from which we can derive H(X'\A) < H(X') with 
equality iff A is independent of X 0. In this last in- 
equality we have used H(X'\A) = '}Z a H{X'\a)p{a), and 
H(X'\a) given similarly as in Eq.(j^). 

Now, upon the application of the measurement, one 
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can define the reduction of entropy of the system condi- 
tionally on the outcome of A by AH a = H(X'\A) — 
H(X), which, obviously, satisfies AHa < AH, and 
H (A) > AH — AHa- In other words, the decrease in 
the entropy of X conditioned on the state of A is con- 
pensated for by the increase in entropy of A. This latter 
quantity represents information that A posses about X. 
Accordingly, the entropy of X given A plus the entropy of 
A is nondecreasing, which is an expression of the second 
law of thermodynamics as applied to interacting systems 
TJ. In a similar line of reasoning, Schack and Caves 
showed that some classical and quantum systems 
can be termed "chaotic" because of their exponential sen- 
sitivity to perturbation, by which they mean that the 
minimal information H (A) needed to keep AHa below a 
tolerable level grows exponentially in time in comparison 
to the entropy reduction AH — AHa- 

It must be stressed that the reduction of entropy of X 
discussed so far is conditional on the outcome of A. By 
assumption, X is not affected by A; as a result, accord- 
ing to an observer who does not know this outcome, the 
entropy of X is unchanged. In order to reduce entropy 
for all observers unconditioned on the state of any ex- 
ternal systems, a direct dynamical action on X must be 
established externally by a controller C whose influence 
on the system is represented by a set of control actions 
x A x 1 triggered by the controller's state c. Mathemat- 
ically, these actions can be modelled by a probability 
transition matrix p{x'\x,c) giving the probability that 
the system in state x goes to state x 1 given that the 
controller is in state c. The specific form of this actua- 
tion matrix will in general depend on the subdynamics 
envisaged in the control process: some of the actions, 
for example, may correspond to control strategies forc- 
ing several initial conditions to a common stable state, 
in which case the corresponding subdynamics is entropy 
decreasing. Others can model uncontrolled transitions 
perturbed by external or internal noise leading to "fuzzy" 
actuation rules which increase the entropy of the system. 
Hence, the systems X and C need not in general model 
a closed system; X, as we already noted, can also be af- 
fected by external systems (e.g., environment) on which 
one has usually no control. However, formally speaking, 
one can always embed any open-system evolution in a 
higher dimensional closed system whose dynamics mim- 
ics a Hamiltonian system. This can be done by supple- 
menting an open system with a set of ancillary variables 
acting as an environment £ in order to construct a global 
volume-preserving transition matrix such that, when the 
ancillary variables are traced out, the reduced transition 
matrix reproduces the dynamics of the system X + C. 

Note that these ancillary variables thus introduced 
need not have any physical significance: they are only 
there for the purpose of simplifying the analysis of the 
evolution of the system. In particular, no control can 
be achieved through the choice of £. Within our model, 
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the control of the system X can only be assured by the 
choice of the control C whereby we can force an ensem- 
ble of transitions leading the system to a net entropy 
change AH. Since the overall dynamics of the system, 
controller and environment is Hamiltonian, Landauer's 
principle immediately implies that if the controller is ini- 
tially uncorrelated with the system (open-loop control), a 
decrease in entropy AH for the system must be compen- 
sated for by an increase in entropy of at least AH for the 
controller and the environment Furthermore, using 
again the concavity property of H, it can be shown that 
the maximum decrease of entropy achieved by a partic- 
ular subdynamics of control variable c is always optimal 
in the sense that no probabilistic mixture of the control 
parameter can improve upon that decrease. Explicitly, 
we have the following theorem (we omit the proof which 
follows simply from the concavity property.) 

Theorem 1. — For open- loop control, the maximum 
value of AH can always be attained for a pure choice 
of the control variable, i.e., with p(c) — 1 and p(c) = 
for all c/c, where c is the value of the controller leading 
to maxAff. Any mixture of the control variables either 
achieves the maximum or yields a smaller value. 

From the standpoint of the controller, one major draw- 
back of acting independently of the state of the sys- 
tem is that often no information other than that avail- 
able from the state of X itself can provide a reason- 
able way to determine which subdynamics are optimal 
or even accessible given the initial state. For this rea- 
son, open-loop control strategies implemented indepen- 
dently of the state of the system or solely on its statis- 
tics usually fail to operate efficiently in the presence of 
noise because of their inability to react or be adjusted 
in time. In order to account for all the possible be- 
haviors of a stochastic dynamical system, we have to 
use the information contained in its evolution by con- 
sidering a closed-loop control scheme in which the state 
of the controller is allowed to be correlated to the ini- 
tial state of X. This correlation can be thought as a 
measurement process described earlier that enables C 
to gather an amount of information given formally in 
Shannon's information theory HQ by the mutual in- 
formation I{X;C) = H(X) + H(C) - H(X,C),whete 
H(X, C) = — J2 X c^t 1 ' c ) l°g-P( a; 7 c ) i s th e joint entropy 
of X and C. Having defined these quantities, we are now 
in position to state our main result which is that the 
maximum improvement that closed-loop can give over 
open-loop control is limited by the information obtained 
by the controller. More formally, we have 

Theorem 2. — The amount of entropy AH c \ ose & that can 
be extracted from any dynamical system by a closed-loop 
controller satisfies 

A-ffdoscd < AH opcn + I(X; C), (3) 

where AH opcn is the maximum entropy decrease that can 
be obtained by open-loop control and I(X; C) is the mu- 



tual information gathered by the controller upon obser- 
vation of the system state. 

Proof. — We construct a closed system by supplement- 
ing an ancilla £ to our previous system X + C. More- 
over, let C and £ be collectively denoted by B with state 
variable B. Since the complete system X + B is closed, 
its entropy has to be conserved, and thus H(X, B) = 
H(X' , B'). Defining the entropy changes of X and B by 
AH = H{X)-H{X') and AH B = H(B')-H(B) respec- 
tively, and by using the definition of the mutual informa- 
tion, this condition of entropy conservation can also be 
rewritten in the form AH = AH B - I(X'; B') + I(X; B) 
jn]]. Now, define AiJ opcn as the maximum amount of 
entropy decrease of X obtained in the open-loop case 
where I(X;C) = I(X;B) = (by construction of £, 
I(X; E) = 0.) From the conservation condition, we hence 
obtain max AH = AH opcn + I(X;B), which is the de- 
sired upper bound for a feedback controller. 

To illustrate the above results, suppose that we con- 
trol a system in a mixture of the states {0,1} using a 
controller restricted to use the following two actions 

(c = 0: x^>x' = x ,.s 
\ c = 1 : x — >> X 1 = NOT X ^ ' 

(in other words, the controller and the system behave like 
a so-called 'controlled-NOT' gate). Since these actuation 
rules simply permute the state of X, H(X') > H(X) 
with equality if we use a pure control strategy or if 
H(X) — -ff m ax = 1 bit, in agreement with our first theo- 
rem. Thus, AH opcn = 0. However, by knowing the actual 
value of x (H(X) bit of information) we can choose C 
to obtain AH = H(X), therefore achieving Eq.(||) with 
equality. Evidently, as implied by this equation, informa- 
tion is required here as a result of the non-dissipative na- 
ture of the actuations and would not be needed if we were 
allowed to use dissipative (volume contracting) subdy- 
namics. Alternatively, no open-loop controlled situation 
is possible if we confine the controller to use entropy- 
increasing actuations as, for instance, in the control of 
nonlinear systems using chaotic dynamics. 

To demonstrate this last statement, let us consider the 
feedback control scheme proposed by Ott, Grebogi and 
Yorke (OGY) |l^| as applied to the logistic map 

x n+1 = rx„(l - x n ), ate [0,1], (5) 

(the extension to more general systems naturally follows). 
The OGY method, specifically, consists of applying to 
Eq.([s]) small perturbations r — > r + Sr n according to 
5r n = —j(x n — x*), whenever x n falls into a region D 
in the vicinity of the target point x* . The gain 7 > 
is chosen so as to ensure stability p3|. For the purpose 
of chaotic control, all the accessible control actions de- 
termined by the values of 5r n , and thereby by the co- 
ordinates x n G D, can be constrained to be entropy- 
increasing for a proper choice of D, meaning that the 
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Lyapunov exponent A(r) associated with any actuation 
indexed by r is such that A(r) > Physically, this 
implies that almost any initial uniform distribution for 
X covering an interval of size e "expands" by a factor 
2 A ( r ) on average after one iteration of the map with pa- 
rameter r p6| . Now, for an open- loop controller, it can 
be readily be shown in that case that no control of the 
state x is possible; without knowing the position x n , a 
controller merely acts as a perturbation to the system, 
and the optimal control strategy then consists of us- 
ing the smallest Lyapunov exponent available so as to 
achieve AH opcn = — X min < 0. Following theorem 2, it is 
thus necessary, in order to achieve a controlled situation 
AH > 0, to have I(X; C) > A m i n using a measurement 
channel characterized by an information capacity of 
at least Amin 

bit per use. 

In the controlled regime (n — > oo), this means specifi- 
cally that if we want to localize the trajectory generated 
by Eq.(Q) uniformly within an interval of size e using a 
set of chaotic actuations, we need to measure x within 
an interval no larger than e:2~ Amin . To understand this, 
note that an optimal measurement of I(X;C) = log a 
bits consists, for a uniform distribution p(x) of size e, in 
partitioning the interval e into a subintervals of size e/a. 
The controller under the partition then applies the same 
actuation rW for all the coordinates of the initial density 
lying in each of the subintervals i, therefore stretching 
them by a factor 2 A ^ ( h In the optimal case, all the 
subintervals are directed toward x* using A m i n and the 
corresponding entropy change is thus 

Ai/cioscd = log e - log 2 Ami "e/a = - A min + log a, (6) 

which is consistent with Eq.(^|) and yields the aforemen- 
tioned value of a for AH = 0. Clearly, this value con- 
stitutes a lower bound for the OGY scheme since not all 
the subintervals are controlled with the same parameter 
r, a fact that we observed in numerical simulations p7| . 

In summary, we have introduced a formalism for study- 
ing control problems in which control units are analyzed 
as informational mechanisms. In this respect, a feedback 
controller functions analogously to a Maxwell's demon 
p8[ , getting information about a system and using that 
information to decrease the system's entropy. Our main 
result showed that the amount of entropy that can be ex- 
tracted from a dynamical system by a controller is upper 
bounded by the sum of the decrease of entropy achiev- 
able in open-loop control and the mutual information be- 
tween the dynamical system and the controller instaured 
during an initial interaction. This upper bound sets a 
fundamental limit on the performance of any controllers 
whose designs are based on the possibilities to accede low 
entropy states and was proven without any reference to 
a specific control system. Hence, its practical implica- 
tions can be investigated for the control of linear, non- 
linear and complex systems (discrete or continuous), as 



well as for the control of quantum systems for which our 
results also apply. For this latter topic, our probabilis- 
tic approach seems particularly suitable for the study of 
quantum controllers. 
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